Support Vector Machine and Generalization

The Support Vector Machine (SVM) has been extended to build up nonlinear classifiers using the kernel trick [1– 3]. As a learning model, it has the best recognition performance among the many methods currently known because it is devised to obtain high performance for unlearned data. The SVM uses linear threshold elements to build up two-classes classifier. It learns linear threshold element parameters based on “margin maximization” from training samples. This paper reviews how to enhance generalization in learning classifiers. The SVM is introduced, then multiple regression analysis (MRA) and logistic regression analysis (LRA) are explained as the statistical methods for building up a classifier with a structure similar to that for the SVM. The same method as used for the SVM can be introduced in both MRA and LRA to enhance performance for unlearned samples. This paper reviews how to enhance generalization in classifier learning and compares the SVM with these methods at the criterion function level[4].